The present invention is related to a system for recognizing a large variety of different object types in colorized three-dimensional (3D) point cloud data generated by a fusion of 3D Light Detection and Ranging (LIDAR) and two-dimensional (2D) color imaging sensor data. While nothing heretofore devised recognizes objects using colorized 3D point cloud data by fusing image sensor data, 3D object recognition systems do exist. For example, a 3D object recognition system is produced by Sarnoff Corporation, located at 201 Washington Road, Princeton, N.J. 08540.
Sarnoff's 3D object recognition system utilizes a coarse-to-fine scheme for object indexing and rotationally invariant spin image features for object representation. The recognition process consists of matching input features with a database of object models using locality sensitivity hashing. Such an approach does not work well if the objects exhibit large intra-class variability. Sarnoff's system also does not utilize context since objects are recognized independently, cueing mechanisms are not provided, and exhaustive search must be done in x, y, z, and scale. In addition, spin images require the estimation of normal vectors on a surface mesh enclosing the object. Such vectors are sensitive to noise and are inaccurate if the sampling density is insufficient.
Previous approaches to recognition of objects in 3D point clouds assumed objects are independent of their surroundings. Such prior art systems have not taken advantage of the hierarchical taxonomies of objects and the relationships of objects with the environment.
Thus, a continuing need exists for an object recognition system that recognizes different object types in colorized 3D point cloud data that considers the hierarchical taxonomies of object and the relationships of the objects with the surrounding environment.